# Self-supervised Pretraining

The first comprehensive benchmark study model for self-supervised learning on 3D medical imaging data

The first comprehensive benchmark study model for self-supervised learning on 3D medical imaging data

A self-supervised foundational model for broadly scalable cardiac applications, trained on 9.1 million 12-lead ECGs covering 164 cardiovascular diseases

BERTurk-Legal is a Transformer-based language model specifically designed for prior case retrieval tasks in the Turkish legal domain.

MoLFormer is a chemical language model pre-trained on 1.1 billion molecular SMILES strings from ZINC and PubChem. This version uses 10% samples from each dataset for training.

VideoMAE is a self-supervised pretrained video model based on Masked Autoencoder (MAE), fine-tuned on the Something-Something V2 dataset for video classification tasks.

RegNetY-64GF feature/backbone model, pretrained using the SEER method on 2 billion random internet images with self-supervised learning

Vision Transformer model pretrained using MSN method, excels in few-shot scenarios

This vision transformer model is pretrained using the MSN method and is suitable for few-shot learning scenarios, particularly for image classification tasks.

VideoMAE is a self-supervised pretraining model for videos based on Masked Autoencoder (MAE), pretrained for 800 epochs on the Something-Something-v2 dataset.

Document image classification model pretrained on IIT-CDIP and fine-tuned on RVL-CDIP, using Transformer architecture

DiT is a Transformer-based document image classification model, pretrained on the IIT-CDIP dataset and fine-tuned on the RVL-CDIP dataset

BEiT is a vision Transformer-based image classification model pretrained in a self-supervised manner on ImageNet-21k and fine-tuned on ImageNet-1k.

BEiT is a vision Transformer-based image classification model, pretrained in a self-supervised manner on ImageNet-21k and fine-tuned on ImageNet-1k.

WavLM is a large-scale self-supervised pretrained speech model developed by Microsoft, pretrained on 16kHz sampled speech audio, suitable for various speech processing tasks.